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DESCRIPTION 



Object- specific segmentation 

The present inwntionidatestothefieldof digital imaging In particular, the present 
invention relates to a method of segmenting an object of interest fromartuiti-^diir^anal 
5 dataset, to an image prooesangdevice and to a computer program for segmenting an object 
of interest from a multi-dimensional 
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Segmentation methods are used to derive geometric models of, for example, organs or bones 
or other objects of interest frommulti^fcrensional datasets, such as volumetric image data, 
such asCT, MR or US images. Such geometric models are required for a variety of medical 
appKcations, or generally in the field of pattern recognition For medical or clinical 
appHcations, an important example is cardiac diagnosis, where geometric models of the 
ventricles and the myocardium of the heart are required, for example, for perfusion analysis, 
will morion analysis and computation of the ejection fraction. Another important clinical 
application is radio-therapy planning (RTP), where the segmentation of multiple organs and 
bones, for example, in the prostate region, is necessary for the diagnosis and/Or the 
detenxrination of the treatment parameters 



Defarmable models are a very general class 
20 3D images. Deformable models are known, 
aL 'IDeformable models in 
(2): 91-108 1996. 



structures in 
from an article of T. Mclnerneyet 
'in Medical Image Analysis, 1 
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30 



Segmentation by deformable models is typically carried out by adapting flexible meshes, 
represented, for example, by triangles or sirrplexes, to the boundaries of the object of interest 
in an image. For this, the model is initially placed near or on the object of interest in the 
image This may be done by a user. Then, coordinates of surface elements of the flexible 
mesh, such as triangles, are iteratively changed until they He on or close to the surface of the 
object of interest. Such a method is described in further detail in J. WeeseetaL 'Shape 
corfitrained deformable models for 3D medical image segmentation "in 17 th International 
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Conferenoe on Information Processing in Medical Imaging (DPMI), pages 380 to 387, 
Daviess CA USA 2001, Springer Vedag 

The optimal adaptation of an initial mesh is found by energy mirrimizatiorv where 
5 maintairdngthediapeof a geometric modd is traded off against ^ 

the object surface in the imags. Feature point detection maybe carried out locally for each 
triangle or simplex by seardiingfor possible object surfaces in the image, for example, for the 
maximum image gradient along a normal of the triangle or simplex. 

1 0 Such segmentation methods may, however, Ml to correctly segment anatomical structures 
with complex and/br ambiguous feature information. One example is the segmentation of 
therertimimixdiatio (RTP), where air maybe present, and the correct 

segmentation of the rectum wall is difficult. 

15 It is an object of the present invention to provide for an improved object segmentation 

A:coiding to an exemplary embodiment of the present invention, the above object maybe 
solved by a method of segmenting an obj ect of interest from a multi-dimensional dataset, 
such as an image, wha^adeformablemoddsurfaceistobe adapted to a surface of the 

20 object. Aicording to an aspect of the present inventiorv object-specific data is acquired, 
which is used during the adaptation of the def ormable surface model to the surface of the 
object. Advantageously, due to the use of object-specific a priori information adaptation of 
the segmentation process for adapting the def ormable surface model to the surface of the 
object , an improved segpnentation maybe provided, where, for example, a rectum wall, 

25 even in the presence of air in the rectum, may b e segmented Furthermore, advantageously, 
the method according to this exemplary embodiment of the present invention may provide 
for an improved segmentation of different obj ects which are located close to each other. 
AJvantageously, a differentiation between those dose objects maybe improved 

30 Azcordingto another exemplary embodiment of the present invention as set forthin daim 
2, the ol^ect-qpecific data 

form of a polygonal rnodd representing the object surface, a point distribution shape model, 
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for example, as described in Cootes et aL 'The use of active shape models for locating 
structures in medical images" in Image and Vision Cornputing. 12(6): pages 355-366, 1994, 
vsWdiisrei^yincc^ratedbyrefermce, an object-specific feature search function, an 
object-specific parameter setting and object-specific material properties Aivantageously, 
5 according to this exemplary errbodirnent of the present invention, a robustness of a model 
based segmentation of collections of anatomical structures such as, for example, in 
radiotherapy planning (RTF) may be increased 

Acording to another exemplary embodiment of the present invention as set forth in claim 
10 3, an object-specific feature search function is applied, which is adapted such that it responds 
to apre-dermedrangeof values selected from a group consisting of a gradient, a gradient 
direction and an intensity range. Alvantageously, returning to the example of the air filled 
rectum, this allows, for example, to apply a different threshold value in the case that an air 
bubble is detected in the rectum, which inherently causes a very steep gradient 

15 

Awarding to another exemplary errbcxEment of the present invention as set forth in claim 

4, the object-specific parameter setting is adapted to control the influence of image features 
and shape constraints, 

20 For example, when segmentingbony structures such as femoral heads or spinal vertebrae, the 
organ variability is limited and the value of the weighting parameter for the internal energy 
controlling the shape deviation from the model can be larger compared to that parameter for 
the soft tissue organs, eg bladder. 

25 Acordingto another exemplary embodiment of the present invention as set forth in claim 

5, the object-specific material properties relate to tissue properties of an organ Such tissue 
properties may, for example, be an elasticity of the tissue or a blood supply in an organ 
region Such tissue properties may, for example, be assigned to the internal nodes of the 
volumetric mesh of the deformable surface model 

30 

Acording to another exemplary embodiment of the present invention as set forth in claim 

6, the organ specific data is acquired by displaying a graphical user interface (GUI) to a user 
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and prompting the user to input such information Then, this input is read and written into 
a memory. Advantageously, this may allow for an interactive input of such information 
during operation and, furthermore, for a later re-use of such information in a "drag and 
drop" style during later operation. 

5 

According to another exemplary emlxxliment of the present invention as set forth in claim 

7, the obj ect-specific data is read from a memory. According to thi§, orgpn specific data may- 
be collected in a memory and stored for later re-use. 

10 According to another exemplary embodiment of the present invention as set forth in claim 

8, the method according to the present invention is an organ segpnentatian method for 
segmenting anatomical structures in medical 

embodiment of the present invention as set forth in claim 9, an image processing device is 
provided, ccraiprising a memory data and an image 

15 processor for segmenting an object of interest from an image. In this image processor, a 
deformable surface model is adapted to a surface of the obj ect by using the obj ect-specific 
data Advantageously, due to the incorporation of object-specific a prior information to the 
segmentation process, the robustness of the model based segmentation of , for example, 
anatomical structures, maybe improved Furthermore, the segmentation results have an 

20 improved reliability, due to the incorporation of the object-specific a priori information in 
the segmentation process or in organ deformation prediction. 

According to another exemplary embodiment of the present invention as set forth in claim 
10, a computer program is provided, allowing for an improved segmentation. The computer 
25 program may be written in any suitable program language, such as C ++ and may be stored 
on a computer readable device, such as a CD-ROM However, the computer program 
according to the present invention may also be presented over a network such as the 
WoridWideWeb, from which it maybe downloaded 

30 It may be seen as the gjst of an exemplary embodiment of the present invention that object- 
specific a priori information is incorporated into the segmentation process According to an 
aspect of the present invention, it may also be incorporated into orgpn deformation 
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prediction In particular, this may be done interactively by prompting the user to input such 
information by displaying a GUI to the user. The input information may then b e stored in a 
memory for later retrieval. Advantageously, this may allow for an activation of such 
information in a drag and drop style 

5 

These and other aspects of the present invention will become apparent from an elucidated 
with reference to the embodiments described hereinafter. 

Exemplary emfoodimmts of the present 
10 reference to the following drawings 

Fig 1 shows a schematic representation of an image processing device according to an 
exemplary embodiment of the present invention, adapted to execute a method according to 
an exemplary embodiment of the present invention 

15 

Fig 2 shows a simplified flowdnart of an exanplary embodiment of a method for operating 
the imagp processing device of Fig. 1 according to the present invention 

Fig 1 shcwsasimpMedscte 

20 processing device in accord In Fig. 1 there is shown a cmtaal 

processing unit (CPU) or image processor 1 for adapting a def ormable model surface to 
surfaces of an object of interest by mesh adaptation The object may also be composed of 
multiple objects. In addition to being conceived to adapt a deformable model surface to the 
object surface, the image processing device depicted in Fig 1 may also be adapted to 

25 detamine or generate a surface model from one or a plurality of trairring models 

The image processor 1 is connected to a memory 1 for storing a multi-dimenaonal dataset. 
Such multi- dimensional dataset s are referred to the in the following as images The image 
processor 1 maybe connectedby a bus system. 3 to a plurality of periphery devices or 
30 input /butput devices which are not dqpictedinFig 1. For example, the image processor 1 
maybe connected to an MR device, aCT device, an ultrasonic scanner, to a plotter or a 
printer or the like via the bus system 3. Furthermore, the image processor 1 is connected to a 
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display such as a computer screen 4 for outfitting segmentation results Furthermore, the 
display maybe used to display a graphical user interface (GUI) to prompt the user to input 
object-specific a priori inforrnation. Furthermore, a keyboard 5 is provided, connected to the 
image procesGor 1, by which a user or operator may interact with tte image processor 1 or 
may input data necessary or desired for the segmentation process 

Fig 2 shovssa amplified flow*art of an exemplary embodiment of a method for operating 
the image processing device depicted inFigl. 

After the start in step SI , the method continues to step S2, where it is determined whether 
the obj ect-specific data is acquired from a memory or a user. In case it is determined in step 
S2 that the object-specific data is acquired from a user, the method continues to step S3. In 
step S3, a GUI is generated by the image processor 3 and output to a user via the display. 
The GUI may prompt the user to input object-specific data For this, the GUI maybe 
adapted as a template, comprising blanks, where the user may input the specific informatiorL 
The specific information is a combination of organ specific a priori knowledge, which is 
incorporated into the subsequent segmentation process According to an exemplary 
embodiment of the present invention, anatomical structures are segmented in medical 
images In such cases, the organ specific a priori knowledge may relate to shape properties, 
for example, in the form of an organ specific shape model which is applied Such organ 
specific shape model may, for example, be a point distribution model (PDM) as described in 
Cootes et al 'The use of active shape models for locating structures in medical images" in 
Image and Vision Computing. 12(6): pages355-366, 1994, which is hereby incorporated by 
reference, consisting of the mean organ shape as w^ as pimcipal variation modes. 

Furthermore, according to an aspect of the present mention, such organ specific a priori 
knowledge may relate to organ specific feature search functions, which are applied to detect 
feature points on the object surface in the image Suitable feature search functions are, for 
example, described in detail in J. Weese et aL 'Shape constrained deformable models for 3D 
image segmentation" in 17* International Conference on Information Processing in Medical 
Imaging (IPMI), pages380 to 387, Davies, CA USA 2001, Springer Verlag which is 
hereby irrorporated by reference Thus, in accordance with an aspect of the present 
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mvention, these organ specific feature sem^iu^o^r^ybe adapted sachet ^ey 
n^spondtoapre^Mned range of values Such valuesrr^forexarrpHbeagracientinthe 
obj ect region of the image, an intensity range or a gradient direction. Furthermore, the organ 
specific a priori knowledge may relate to organ specific parameter settings to control the 

^corisfxairttsL 



For example, when segmenting bony structures such as femoral heads or spinal vertebrae the 
coganvar^tyislimitea and the valued 
centrally tte 

the soft tissue organs, eg bladder. 

organ specific knoWedge, as indicated above, n^ath^beir^byausercrreadrroma 
memory. Such material properties may relate to, for example, an elasticity of the respective 
organ tissue Such tissue properties may, for example, be assigned to the total amounts of a 
volumetric mesh. 



All of the abovelistedorgan specific apriori data rr^ also be used for tasks cte than 
segmentation, for example, for organ c3eformation prediction and 4D RTP. 

After an operator or user ha^ 
S4,v^themforrr^onir^bytr e user OT 

to step S5, vtetheebject-spedfic data input and read in steps S3 andS4 is stored i 
memory as object-specific data Then, the method continues to step S6. 



lina 



mc^seitv^detetminedmstepSZ that 

method continues to step S6. In step S6, the suitable deformable surface model 
corresponding to the organ to b e segmented, is loaded The deformable surface model may 
bespedfic^yadaptedto 

is to be segmented, a corresponding prostate region deformable surface model is loaded 
Then, the method continues to step S7, v^ere the obj ect-spedfic data is retrieved from the 
memory. Then, in the subsequent step S3, the deformable surface model is iteratively 



-8- 



PHDE030250EP-P 



adapted to the surface of the object by using the object-specific data as described mth 
reference to steps S3 and S4. The generation of a suitable deformable surface model and the 
adaptation of the surface 

Weese et al 'Shape constrained deformable models for 3D image segmentation" in 1 7 th 
5 International Conference on Information Processing in Medical Imaging (IPMI), pages 380 
to 387, Davie£> CA USA 2001, Springs: Vedag, which is hereby incorporated by reference. 

According to an aspect of this exemplary embodiment of the present invention, during the 
feature search, points vHch do not comply to a search profile with the properties of the 
10 respective organ (eg the g^ay valuer the value and direction of the gradient etc) are ignored 
and not taken into account . Eg, for the bladder, the interval of the gray values differs from 
the interval of the femur wHch maybe used accessing to an aspect of the present invention. 

Then, the method continues to step S9, whene the segmentation result is output. After the 
1 5 output of the segmentation result in step S9, the method continues to step SI 0, where it 
ends. 

Advantageously, the above described method allows to further increase the robustness of the 
model based segmentation of collections of anatomical structures. In particular, it allows for 
20 an improved radiotherapy planning, where a segmentation of a collection of several 

anatomical structures, which represent a target volume an risk organs is required Asset forth 
above, this may in particular be achievedby incorporating a priori object of organ specific 
information into the segenantation process. 

25 Furthermore, due to the fact that the organ specific data may b e interactively input by a user 
and subsequently stared in a memory, either an interactive process may be provided, or a 
semi-automatic process, where the respective organ specific information maybe presented to 
a user, such that the user may activate such pre-stored information in a drag and drop style. 

30 In particular, such organ specific information may be presented to the user inavraythat pre- 
stared or^ specific data ^ 
alter accordingly. 
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^part foam providingaAoy robust model based segmentation with an improved acc 
the above method may allow to oanaderably reduce the time required for treatment 
planning, in particular in RTP. 
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CLAIMS 



1 . Method of segmenting an object of interest from a multi-dimensional dataset, 
vfaecein a deformable surface model is to be adapted to a surface of the obj ect, method 
comprising the steps of: 

acquiring object-specific data; 

adapting the defbmnable surface modd to the su^ 
object-specific data 

2. The method of claim 1, 

therein the object-specific data is sdectedfrcm the gcoup consist^ 

the from of an object model a point distribution model, an object-specific feature search 

function, an object-specific parameter setting and object-specific material properties 

3. The method of claim 2, 

v^herein the obj ect-^ecific feature search function is adapted to a predefined range of values 
selected from the group consisting of a gradient, a direction of a gradient and an intensity 
rang* 

4. The method of 

vvhereinthedtg setting is adapted to control an influence of image 

features and shape ranslxaints 

5. The method of claim 2, 
^hetantheo^ 

are. 
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6. The method of claim 1, \\herein the step of acquiring object-specific data 
comprises the steps of: 

displaying a graphical user interface on a display prompting a user to input obj ect related 
information; 

5 receiving a crarespmding data input from an input device; 
storing the data input as object-specific data in a memory. 

7. The method of claim 1, v^herein the step of acquiring object-specific data 
comprises the steps of: 

10 reading the object-specific data from, a memory. 

8. The method of claim. 1, 

wherein the method is an org^n segpnentation method for segmenting anatomical structures 
in medical images 

15 

9. Image processing device, comprising 

a memory for storing acquired object-specific data; and 

an image processor for segpnentirtg an object of interest from an image, wherein a deformable 
surface model is adapted to a surface of the obj ect by using the obj ect-specific data 

20 

10. Computer program for segmenting an obj ect of interest from a multi- 
dimensional datas^ 

obj ect, wherein the computer program causes a processor to perform the following steps 
when the computer is executed on the processor. 
25 acquiring object-specific data; 

adapting the deformable surface model to the surface of the obj ect by using the 
object-specific data 
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ABSTRACT 

Object- specific segmentation 

The invention relates to the field of efficient segmentation of collections of anatomical 
structures in medical imaging. For example, in radiotherapy planning, the segmentation of a 

5 collection of several anatomical structures, which represent the target volume in risk organs is 
required When using model based segmentation, organ models represented by flexible 
surfaces are adapted to the boundaries of the object of interest According to an aspect of the 
present invention, object- specific a priori information is incorporated in the segmentation 
process, which allows to provide for an improved segmentation. Furthermore, the 

10 segmentation process according to the present invention, may have an improved robustness, 
also the time required for the segmentation may be reduced. 
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S3 
S4 

S5 



user 



displaying a GUI 




reading input 



storing input as object- 
specific data 



loading deformable surface model 



memory 



S6 



retrieving object-specific data from memory S7 



iteratively adapting the deformable surface 
model to the surface of the object by using the 
object-specific data 
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outputting segmentation result 
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